Face recognition apparatus and method using plural face images

ABSTRACT

A face recognition apparatus and method using plural face images includes detecting a reference face image from an image input for registration in advance, calculating plural face feature information from a frontal face image if the detected reference face image includes the frontal face image, generating plural compared face images using the calculated plural face feature information, and determining whether the input face image matches the generated plural compared face images by comparing the input face image with the plural compared face images. Face images having diverse points of view are generated through single user registration, and thus the face recognition ratio can be heightened.

PRIORITY

This application claims priority under 35 U.S.C. §119(a) to anapplication entitled “Face Recognition Apparatus And Method Using PluralFace Images” filed in the Korean Intellectual Property Office on Mar. 9,2010, and assigned Serial No. 10-2010-0020857, the entire disclosure ofwhich is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to an apparatus and method forrecognizing a face from an image, and more particularly to an apparatusand method for recognizing a face from an input image using plural faceimages.

2. Description of the Related Art

With the development of information technology within society, identityconfirmation technologies for identifying a person, have becomeimportant. In order to improve personal information protection andidentity confirmation using a computer, biometric technologies usinghuman physiological features are frequently being researched.

Among the biometric technologies, face recognition technology has beenevaluated as a convenient and competitive biometric technology due toits advantage of confirming the identity of a user in a non-contactmanner, unlike recognition technologies that require user operation oraction, such as fingerprint recognition, iris recognition, and the like.

In general, according to face recognition technology, face images usedto recognize faces are registered in advance, and a face is recognizedby comparing a registered face image with a face image detected from aninput image.

Such face recognition technology has been widely used in diverse fieldsand applications, such as summarizing a moving image using faceinformation, identification, Human Computer Interface (HCI) imagesearch, security, monitoring systems, and the like, and applied as oneof the core technologies fbr a multimedia database search.

As described above, according to the face recognition method in therelated art, face images to be used for face recognition are registeredin advance, and a face is recognized by comparing an image input forface recognition with a registered face image.

However, the face recognition method in the related art has the problemthat if a face image, which is registered by a user in a specified pointof view, is input in a different position from the face image registeredfor face recognition, the face recognition may not be accuratelyperformed.

Another problem of the face recognition method in the related art isthat a user should register a face image several times when the userregisters the face image to be used for the face recognition, and if theuser registers the face image only once, the number of images to becompared with the input image is decreased to lower the face recognitionratio.

SUMMARY OF THE INVENTION

Accordingly, the present invention has been made to solve theabove-mentioned problems occurring in the related art. The presentinvention provides a face recognition apparatus and method throughgeneration of face images in several positions when a user registers thecorresponding face image.

In accordance with an aspect of the present invention, there is provideda face recognition apparatus using plural face images, which includes aface detection unit detecting a reference face image from an image inputfor registration in advance; a face feature point calculation unitcalculating plural face feature information from a frontal face image ifthe detected reference face image is the frontal face image; a comparedface generation unit generating plural compared face images using thecalculated plural face feature information; and a face recognition unitdetermining whether the input face image matches the generated pluralcompared face images by comparing the input face image with the pluralcompared face images.

In accordance with another aspect of the present invention, there isprovided a face recognition method using plural face images, whichincludes detecting a reference face image from an image input forregistration in advance; calculating plural face feature informationfrom a frontal face image if the detected reference face image is thefrontal face image; generating plural compared face images using thecalculated plural face feature information; and determining whether theinput face image matches the generated plural compared face images bycomparing the input face image with the plural compared face images.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of the presentinvention will be more apparent from the following detailed descriptiontaken in conjunction with the accompanying drawings, in which:

FIG. 1 is a diagram illustrating the configuration of a face recognitionapparatus according to an embodiment of the present invention;

FIG. 2 is a diagram illustrating plural compared face images generatedby a compared face generation unit according to an embodiment of thepresent invention;

FIG. 3 is a flowchart illustrating a process for registering a faceimage that is used for face recognition in a face recognition apparatusaccording to an embodiment of the present invention; and

FIG. 4 is a flowchart illustrating a process for recognizing a face ofan input image in a face recognition apparatus according to anembodiment of the present invention.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE PRESENT INVENTION

Hereinafter, embodiments of the present invention will be described withreference to the accompanying drawings. In the following description ofthe present invention, a detailed description of known functions andconfigurations incorporated herein will be omitted when it may make thesubject matter of the present invention rather unclear.

FIG. 1 is a diagram illustrating the configuration of a face recognitionapparatus according to an embodiment of the present invention.

A face recognition apparatus according to an embodiment of the presentinvention includes a face detection unit 100, a frontal facedetermination unit 110, a face feature point calculation unit 120, acompared face generation unit 130, a memory unit 140, and a facerecognition unit 150.

The face detection unit 100 determines whether a face image is includedin an image input for registration in advance, and if the face image isincluded, extracts the corresponding face image. Here, the facedetection unit 100 performs a face detection operation using a generalface detection method, where a face extraction technology using acontour of a face, skin color, skin texture, template, and the like, maybe used. For example, the face detection unit 100 may perform face studyas it scans plural face images, and may detect a face image from theinput image through the performed face study.

The frontal face determination unit 110 determines whether the detectedface image is a frontal face image that is seen from the frontal or anon-frontal face image that is not seen from the front. The frontal facedetermination unit 110 determines the front of the face in diversemethods. Specifically, the frontal face determination unit 110 performsa frontal face study by scanning plural frontal face images, anddetermines that the face image detected through the performed studyrepresents the frontal face. That is, a frontal face image is one inwhich at least the eyes, nose, mouth, and ears are visible.

If the determined face is the front of the face, the frontal facedetermination unit 110 outputs the determined face image to the featurepoint calculation unit 120, while if the determined face is thenon-frontal face, the frontal face determination unit 110 detects eyeposition information from the corresponding face image, and then storesthe detected eye position information in the memory unit 140.

The face feature point calculation unit 120 calculates positioninformation of the eyes, nose, and mouth from the frontal face image inthe advance registration process, and then outputs the calculatedposition information to the compared face generation unit 130.

Also, the face feature point calculation unit 120 calculates the facefeature points from the plural compared face images generated by thecompared face generation unit 130, and calculates compared feature pointvector values for the calculated face feature points.

Thereafter, the face feature point calculation unit 120 calculates theobject face feature points such as eyes from the face image that isdetected when a process for the face recognition is performed,calculates the feature point vector values for the calculated facefeature points, and outputs the calculated feature point vector valuesto the face recognition unit 150.

The compared face generation unit 130 generates plural compared faceimages using the calculated face feature points. Specifically, thecompared face generation unit 120 generates the plural compared faceimages that are seen in several positions, such as in upward, downward,left, and right directions, using the calculated position information ofthe eyes, the nose, and the mouth. The plural compared face image set asabove may be illustrated as in FIG. 2.

The compared face generation unit 120 divides the frontal face imageinto two regions based on a predetermined symmetrical axis, reduces theimage size of one of the divided face images, and increases the imagesize of the other of the divided face images.

For example, the compared face generation unit 120 may set a face thatsees in the right direction by setting a center vertical axis of thefrontal face image as a first symmetrical axis using the calculatedposition information of the eyes, the nose, and the mouth, decreasingthe width of the left-side face image, and increasing the width of theright-side face image around the first symmetrical axis as representedby a reference numeral 200 in FIG. 2. Also, the compared face generationunit 120 may set a face that faces in the left direction by decreasingthe width of the right-side of the face image, and increasing the widthof the left-side of the face image around the first symmetrical axis asrepresented by a reference numeral 210 in FIG. 2.

Also, the compared face generation unit 120 may set a face that faces inthe upward direction by setting a center horizontal axis that crossesthe center vertical axis as a second symmetrical axis, decreasing thewidth of the upper-side of the face image, and increasing the width ofthe lower-side of the face image around the second symmetrical axis asrepresented by a reference numeral 220 in FIG. 2. Also, the comparedface generation unit 120 may set a face that faces in the downwarddirection by increasing the width of the upper-side of the face image,and decreasing the width of the lower-side of the face image around thesecond symmetrical axis as represented by a reference numeral 230 inFIG. 2.

In addition, the compared face generation unit 120 may set compared faceimages so that the frontal face image is seen as faces that face in ninedirections as illustrated in FIG. 2.

In another example, the compared face generation unit 120 may generatethe compared face images as illustrated in FIG. 2, by generating athree-dimensional (3D) face image using the calculated positioninformation of the eyes, the nose, and the mouth, and capturing the faceimage so that the generated 3D face image faces in the upward, downward,left, and right directions.

The memory unit 140 stores plural face feature information by charactercategories to correspond to plural character categories.

Specifically, the memory unit 140 stores the face feature point vectorvalues for the eye position information calculated from the non-frontalimage as the face feature information of a specified person, and storesthe plural compared feature point vector values calculated by the facefeature point calculation unit 120 to correspond to the users.

The face recognition unit 150 determines whether the face comparedfeature point vector values pre-stored by users in the memory unit 140matches the calculated object feature point vector values by comparingthe feature point vector values with each other when the facerecognition process is performed. Thereafter, the face recognition unit150 calculates the maximum correlation vector value having the largestcorrelation among the compared feature point vector values of the users.For example, the face recognition unit 150 may calculate the objectfeature point vector values having the smallest difference value byusers among respective different values between the plural comparedfeature point vector values corresponding to the users and the objectfeature point vector values as the maximum correlation vector values.

Thereafter, the face recognition unit 150 confirms a person whocorresponds to the maximum correlation vector value having the largestvalue among the calculated maximum correlation vector values by users,and outputs the confirmed person as the resultant value of the facerecognition.

As described above, according to the present invention, since a user cangenerate face images in diverse face directions through oneregistration, the face recognition ratio can be heightened and theresources and time required to perform the face recognition can bereduced.

FIG. 3 is a flowchart illustrating a process for registering a faceimage that is used for face recognition in a face recognition apparatusaccording to an embodiment of the present invention.

Referring to FIG. 3, if an image is input in Step 300, the facedetection unit 100 determines whether the face image is detected fromthe input image in Step 310, and if the face image is detected, the facedetection unit 100 proceeds to Step 320, while if not, the facedetection unit 100 terminates the face detection.

In Step 320, the frontal face determination unit 110 determines whetherthe detected face image is a frontal face image, and if the detectedface image is the frontal face image, the frontal face determinationunit 110 proceeds to Step 340, while if not, the frontal facedetermination unit 110 proceeds to Step 330.

In Step 330, the face feature point calculation unit 120 detects the eyeposition information from the detected face image, calculates comparedfeature point vector values for the detected eye position information,and then proceeds to Step 370.

The face feature point calculation unit 120, which has proceeded fromStep 320 to Step 340, detects position information of the eyes, nose,and mouth from the frontal face image, and outputs the detected positioninformation to the compared face generation unit 130.

In Step 350, the compared face generation unit 130 generates pluralcompared face images using the detected position information of theeyes, nose, and mouth, and outputs the plural compared face images tothe feature point calculation unit 120. At this time, as describedabove, the compared face generation unit 130 generates plural comparedface images that are seen in several positions, such as in upward,downward, left, and right directions, using the calculated positioninformation of the eyes, nose, and mouth. For example, the compared facegeneration unit 130 can generate plural compared faces by dividing thefrontal face image into two regions based on a predetermined symmetricalaxis, reducing the image size of one of the divided face images, andincreasing the image size of the other of the divided face images.

In Step 360, the face feature point calculation unit 120 detects eyeposition information from the generated plural compared face images, andcalculates the compared feature point vector values for the detected eyeposition information. Here, the compared feature point vector valuesmean values that are compared with the feature point vector values thatcorrespond to the input face image when the face recognition operationis performed. Although it is exemplified that the eye positioninformation is extracted, the feature point vector values for the facefeature information such as a nose or a mouth on the face may becalculated in addition to the eye position information.

In Steps 360 and 330, the feature point calculation unit 120, which hasproceeded to Step 370, makes the calculated plural compared featurepoint vector values correspond to a character category corresponding tothe detected face, stores the plural compared feature point vectorvalues in the memory unit 140, and then terminates the face registrationprocess.

As described above, according to the present invention, since a user cangenerate face images in diverse face directions through oneregistration, the face recognition ratio can be heightened and theresources and time required to perform the face recognition can bereduced.

FIG. 4 is a flowchart illustrating a process for recognizing a face ofan input image in a face recognition apparatus according to anembodiment of the present invention.

Referring to FIG. 4, if an image is input in Step 400, the facedetection unit 100 determines whether the face image is detected fromthe input image in Step 410, and if the face image is detected, the facedetection unit 100 proceeds to Step 420, while if not, the facedetection unit 100 terminates the face detection.

In Step 420, the face feature point calculation unit 120 determineswhether the eye position information is detected from the detected faceimage, and if the eye position information is detected, the face featurepoint calculation unit 120 proceeds to Step 430, while if not, the facefeature point calculation unit 120 terminates the face recognitionoperation.

In Step 430, the face feature point calculation unit 120 calculates andoutputs the object feature point vector values for the detected eyeposition information to the face recognition unit 150.

In Step 440, the face recognition unit 150 compares the plural comparedfeature point vector values that correspond to the respective usersstored in the memory unit 140 with the calculated object feature pointvector value.

In Step 450, the face recognition unit 150 detects the compared featurepoint vector values having the largest correlation with the objectfeature point vector value among the compared feature point vectorvalues that correspond to the users.

For example, in the case where plural compared feature point vectorvalues for 20 users in total are pre-stored, the face recognition unit150 detects the compared feature point vector value having the largestcorrelation with the object feature point vector value among the pluralcompared feature point vector values which are stored to correspond to afirst person. Thereafter, the face recognition unit 150 detects thecompared feature point vector values having the largest correlation withthe object feature point vector value among the plural compared featurepoint vector values stored to correspond to a second person to twentiethperson, respectively.

In Step 460, the face recognition unit 150 selects a person whocorresponds to the compared feature point vector value having themaximum value among the compared feature point vector values calculatedby users, and outputs the person selected in Step 470 as the facerecognition output value of the input image.

As described above, according to the present invention, since faceimages in diverse directions are generated through a frontal face imageregistered in advance and face recognition is performed using thegenerated face images, the face is recognized from the face images atdiverse angles, and thus a high face recognition ratio can be obtained.

Also, since an additional operation such as calculation of facedirections and complicated coefficient generation is not performed whenface recognition is performed, the time and resources required toperform face recognition can be reduced and the performance of the facerecognition apparatus can be heightened.

While the operation and configuration of the face recognition apparatusand method using plural face images have been shown and described withreference to certain embodiments thereof, it will be understood by thoseskilled in the art that various changes in form and details may be madetherein without departing from the spirit and scope of the invention asdefined by the appended claims.

1. A face recognition apparatus using plural face images, comprising: aface detection unit detecting a reference face image from an image inputfor registration in advance; a face feature point calculation unitcalculating plural face feature information from a frontal face image ifthe detected reference face image includes the frontal face image; acompared face generation unit generating plural compared face imagesusing the calculated plural face feature information; and a facerecognition unit determining whether the input face image matches thegenerated plural compared face images by comparing the input face imagewith the plural compared face images.
 2. The face recognition apparatusas claimed in claim 1, further comprising a frontal face determinationunit determining whether the detected reference face image includes afrontal face image.
 3. The face recognition apparatus as claimed inclaim 2, further comprising a memory unit storing plural face featureinformation by character categories to correspond to plural charactercategories; wherein the face feature point calculation unit detects atleast one position information of an eye, a nose, and a mouth from thefrontal face image.
 4. The face recognition apparatus as claimed inclaim 3, wherein the compared face generation unit generates the pluralcompared face images by dividing the frontal face image into two regionsbased on a predetermined symmetrical axis using the calculated positioninformation, reducing the image size of one of the divided face images,and increasing the image size of the other of the divided face images.5. The face recognition apparatus as claimed in claim 4, wherein theface feature point calculation unit detects face feature informationfrom the generated plural compared face images, calculates comparedfeature information vector values for the calculated face featureinformation, and stores the calculated compared feature informationvector values in the memory unit to make the calculated compared featureinformation vector values correspond to the character category thatcorresponds to the detected face image.
 6. The face recognitionapparatus as claimed in claim 5, wherein the face feature pointdetection unit detects object face feature information from the detectedface image, and calculates an object feature point vector value for thedetected object face feature information.
 7. The face recognitionapparatus as claimed in claim 6, wherein the face recognition unitsearches for the compared feature point vector value having the largestcorrelation with the object feature point vector value by charactercategories by comparing the plural compared feature point vector valuesstored in the memory unit with the calculated object feature pointvector value, selects the compared feature point vector value having themaximum correlation among the compared feature point vector valuessearched by character categories, and outputs a person who correspondsto the selected compared feature point vector value as the result ofrecognition.
 8. A face recognition method using plural face images,comprising the steps of: detecting a reference face image from an imageinput for registration in advance; calculating plural face featureinformation from a frontal face image if the detected reference faceimage includes the frontal face image; generating plural compared faceimages using the calculated plural face feature information; anddetermining whether the input face image matches the generated pluralcompared face images by comparing the input face image with the pluralcompared face images.
 9. The face recognition method as claimed in claim8, further comprising determining whether the detected face imageincludes a frontal face image.
 10. The face recognition method asclaimed in claim 9, further comprising storing plural face featureinformation by character categories to correspond to plural charactercategories.
 11. The face recognition method as claimed in claim 10,wherein calculating the plural face feature information includesdetecting at least one position information of an eye, a nose, and amouth from the frontal face image.
 12. The face recognition method asclaimed in claim 11, wherein generating the plural compared face imagescomprises: dividing the frontal face image into two regions based on apredetermined symmetrical axis using the calculated positioninformation; and reducing the image size of one of the divided faceimages, and increasing the image size of the other of the divided faceimages.
 13. The face recognition method as claimed in claim 12, furthercomprising: detecting object face feature information from the detectedface image after generating the plural compared face images; andcalculating an object feature point vector value for the detected objectface feature information.
 14. The face recognition method as claimed inclaim 13, wherein determining whether the input face image matches thegenerated plural compared face images comprises: comparing thepre-stored plural compared feature point vector values with thecalculated object feature point vector value; searching for the comparedfeature point vector value having the largest correlation with theobject feature point vector value by character categories in accordancewith the result of comparison; selecting the compared feature pointvector value having the maximum correlation among the compared featurepoint vector values searched by character categories; and outputting aperson who corresponds to the selected compared feature point vectorvalue as the result of recognition.